A community-of-practice-based evaluation methodology for knowledge intensive computational methods and its application to multimorbidity decision support.

OBJECTIVE: The study has dual objectives. Our first objective (1) is to develop a community-of-practice-based evaluation methodology for knowledge-intensive computational methods. We target a whitebox analysis of the computational methods to gain insight on their functional features and inner workings. In more detail, we aim to answer evaluation questions on (i) support offered by computational methods for functional features within the application domain; and (ii) in-depth characterizations of the underlying computational processes, models, data and knowledge of the computational methods. Our second objective (2) involves applying the evaluation methodology to answer questions (i) and (ii) for knowledge-intensive clinical decision support (CDS) methods, which operationalize clinical knowledge as computer interpretable guidelines (CIG); we focus on multimorbidity CIG-based clinical decision support (MGCDS) methods that target multimorbidity treatment plans. MATERIALS AND METHODS: Our methodology directly involves the research community of practice in (a) identifying functional features within the application domain; (b) defining exemplar case studies covering these features; and (c) solving the case studies using their developed computational methods-research groups detail their solutions and functional feature support in solution reports. Next, the study authors (d) perform a qualitative analysis of the solution reports, identifying and characterizing common themes (or dimensions) among the computational methods. This methodology is well suited to perform whitebox analysis, as it directly involves the respective developers in studying inner workings and feature support of computational methods. Moreover, the established evaluation parameters (e.g., features, case studies, themes) constitute a re-usable benchmark framework, which can be used to evaluate new computational methods as they are developed. We applied our community-of-practice-based evaluation methodology on MGCDS methods. RESULTS: Six research groups submitted comprehensive solution reports for the exemplar case studies. Solutions for two of these case studies were reported by all groups. We identified four evaluation dimensions: detection of adverse interactions, management strategy representation, implementation paradigms, and human-in-the-loop support. Based on our whitebox analysis, we present answers to the evaluation questions (i) and (ii) for MGCDS methods. DISCUSSION: The proposed evaluation methodology includes features of illuminative and comparison-based approaches; focusing on understanding rather than judging/scoring or identifying gaps in current methods. It involves answering evaluation questions with direct involvement of the research community of practice, who participate in setting up evaluation parameters and solving exemplar case studies. Our methodology was successfully applied to evaluate six MGCDS knowledge-intensive computational methods. We established that, while the evaluated methods provide a multifaceted set of solutions with different benefits and drawbacks, no single MGCDS method currently provides a comprehensive solution for MGCDS. CONCLUSION: We posit that our evaluation methodology, applied here to gain new insights into MGCDS, can be used to assess other types of knowledge-intensive computational methods and answer other types of evaluation questions. Our case studies can be accessed at our GitHub repository (https://github.com/william-vw/MGCDS).

Supporting physicians in the coordination of distributed execution of CIGs to treat comorbid patients.

Clinical Practice Guidelines (CPGs) encode the "best" medical practices to treat patients affected by a specific disease and are widely used in the medical practice. Starting from the '90s', several Computer-Interpretable Guideline (CIG) systems have been devised to provide physicians with CPG-based decision support. CPGs (and CIGs) are devoted to provide evidence-based recommendations for one specific disease. In order to support the treatment of patients affected by multiple diseases (i.e., comorbid patients), challenging additional tasks have to be addressed, such as (i) the detection of the interactions between CIG actions, (ii) their management, and, finally, (iii) the "merge" or conciliation of the CIGs. Several CIG approaches have been recently extended in order to face (at least one of) such challenging problems, and one of them is GLARE. However, besides the solutions to tasks (i)-(iii) above, the "run-time" support to physicians treating a comorbid patient requires additional capabilities, to support the distribution of the management of interactions and of the execution of CIGs among different physicians. In this paper, we propose a general framework, based on GLARE and GLARE-SSCPM, to provide such additional capabilities.

Conformance analysis for comorbid patients in Answer Set Programming.

The treatment of comorbid patients is a hot problem in Medical Informatics, since the plain application of multiple Computer-Interpretable Guidelines (CIGs) can lead to interactions that are potentially dangerous for the patients. The specialized literature has mostly focused on the "a priori" or "execution-time" analysis of the interactions between multiple Computer-Interpretable Guidelines (CIGs), and/or CIG "merge". In this paper, we face a complementary problem, namely, the a posteriori analysis of the treatment of comorbid patients. Given the CIGs, the history of the status of the patient, and the log of the clinical actions executed on her, we try to explain the actions executed on the patient (i.e., the log) in terms of the actions recommended by the CIGs, of their potential interactions, and of the possible ways of managing each such interaction, pointing out (i) deviations from CIG recommendations not explained in terms of interaction management (if any) and (ii) unmanaged interactions (if any). Our approach is based on Answer Set Programming, and, to face realistic problems, devotes specific attention to the temporal dimension.

Supporting the distributed execution of clinical guidelines by multiple agents.

Clinical guidelines (GLs) are widely adopted in order to improve the quality of patient care, and to optimize it. To achieve such goals, their application on a specific patient usually requires the interventions of different agents, with different roles (e.g., physician, nurse), abilities (e.g., specialist in the treatment of alcohol-related problems) and contexts (e.g., many chronic patients may be treated at home). Additionally, the responsibility of the application of a guideline to a patient is usually retained by a physician, but delegation of responsibility (of the whole guideline, or of a part of it) is often used\required (e.g., delegation to a specialist), as well as the possibility, for a responsible, to select the executor of an action (e.g., a physician may retain the responsibility of an action, but delegate to a nurse its execution). To manage such phenomena, proper support to agent interaction and communication must be provided, providing agents with facilities for (1) treatment continuity (2) contextualization, (3) responsibility assignment and delegation (4) check of agent "appropriateness". In this paper we extend GLARE, a computerized GL management system, to support such needs. We illustrate our approach by means of a practical case study.

Representing and querying now-relative relational medical data.

Temporal information plays a crucial role in medicine. Patients' clinical records are intrinsically temporal. Thus, in Medical Informatics there is an increasing need to store, support and query temporal data (particularly in relational databases), in order, for instance, to supplement decision-support systems. In this paper, we show that current approaches to relational data have remarkable limitations in the treatment of "now-relative" data (i.e., data holding true at the current time). This can severely compromise their applicability in general, and specifically in the medical context, where "now-relative" data are essential to assess the current status of the patients. We propose a theoretically grounded and application-independent relational approach to cope with now-relative data (which can be paired, e.g., with different decision support systems) overcoming such limitations. We propose a new temporal relational representation, which is the first relational model coping with the temporal indeterminacy intrinsic in now-relative data. We also propose new temporal algebraic operators to query them, supporting the distinction between possible and necessary time, and Allen's temporal relations between data. We exemplify the impact of our approach, and study the theoretical and computational properties of the new representation and algebra.

Temporal detection and analysis of guideline interactions.

BACKGROUND: Clinical practice guidelines (CPGs) are assuming a major role in the medical area, to grant the quality of medical assistance, supporting physicians with evidence-based information of interventions in the treatment of single pathologies. The treatment of patients affected by multiple diseases (comorbid patients) is one of the main challenges for the modern healthcare. It requires the development of new methodologies, supporting physicians in the treatment of interactions between CPGs. Several approaches have started to face such a challenging problem. However, they suffer from a substantial limitation: they do not take into account the temporal dimension. Indeed, practically speaking, interactions occur in time. For instance, the effects of two actions taken from different guidelines may potentially conflict, but practical conflicts happen only if the times of execution of such actions are such that their effects overlap in time. OBJECTIVES: We aim at devising a methodology to detect and analyse interactions between CPGs that considers the temporal dimension. METHODS: In this paper, we first extend our previous ontological model to deal with the fact that actions, goals, effects and interactions occur in time, and to model both qualitative and quantitative temporal constraints between them. Then, we identify different application scenarios, and, for each of them, we propose different types of facilities for user physicians, useful to support the temporal detection of interactions. RESULTS: We provide a modular approach in which different Artificial Intelligence temporal reasoning techniques, based on temporal constraint propagation, are widely exploited to provide users with such facilities. We applied our methodology to two cases of comorbidities, using simplified versions of CPGs. CONCLUSION: We propose an innovative approach to the detection and analysis of interactions between CPGs considering different sources of temporal information (CPGs, ontological knowledge and execution logs), which is the first one in the literature that takes into account the temporal issues, and accounts for different application scenarios.

Air Pollution Impact on Pregnancy Outcomes in Como, Italy.

OBJECTIVE: This retrospective observational study investigates the association between maternal exposure to air pollutants and pregnancy adverse outcomes in low urbanization areas. METHODS: We used multivariate regression analysis to estimate, in the Como province (2005-2012), the effects of NO(x), NO2, SO2, O3, CO, and PM10 on low birth weight (LBW), babies small for gestational age (SGA), and preterm birth (PTB). RESULTS: PTB was inversely associated with high (5.5 µg/m³) exposure to SO2 (adjusted odds ratio [aOR] = 0.74, 95% confidence interval [95% CI] = 0.58-0.95) and to CO (1.8 mg/m³, aOR = 0.84, CI = 0.72-0.99). PTB risk increased with second trimester exposure to NO(x) (118.3 µg/m³, aOR = 1.53, CI = 1.25-1.87), while LBW risk increased with third trimester PM10 (56.1 µg/m³, aOR = 1.44, CI = 1.03-2.02). SGA was inversely associated with third trimester NO(x) (115.8 µg/m³, aOR = 0.89, CI = 0.79-0.99). CONCLUSIONS: Exposure to SO2 and CO seems to postpone delivery: a longer gestation could compensate for maternal hypoxemic-hypoxic damage.